Dynamics Based Control: Structure

In this paper we introduce a novel approach to continual planning and control, called Dynamics Based Control (DBC). The approach is similar in spirit to the Actor-Critic [6] approach to learning and estimation-based differential regulators of classical control theory [13]. However, DBC is not a learning algorithm, nor can it be subsumed within models of standard control theory. We provide a general framework for applying DBC to discrete Markovian environments, and discuss the key differences between it and a popular alternative for this type of environment — Partially Observable Markov Decision Processes (POMDPs). We then show how a recently developed control scheme based on Extended Markov Tracking (EMT) [9, 10] can be seen as a suboptimal algorithm within the DBC framework, and discuss EMT’s limitations relative to the general DBC approach.